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SUMMARY 


< 2/6 3 9 


Reduced precision, defined as the use of computer words whose precision 
is less than the maximum single precision length of 27 binary bits, is 
applied to the lunar midcourse navigation and guidance scheme previously 
reported in NASA publications TR R -135 and TN D-1208. With the same computer 
program, the evaluation in this report was based on the outbound leg of an 
earth-moon trajectory. A fixed schedule of 45 observations and 3 velocity cor' 
rections was executed each time the trajectory was traversed with the preci- 
sion reduced in one or more of the following areas: (l) the estimated 

trajectory, ( 2 ) the optimal filter, ( 3 ) the space-angle computations, (4) the 
transition matrix, or ( 5 ) the guidance equations. This procedure was followed 
down to the level of precision at which the estimated trajectory could no 
longer be integrated numerically. 


The mathematical operations embodied in the optimal filter are shown to 
be essentially unaffected by the reduced precision over the range studied. 

This invariance, as determined by the behavior of the covariance matrix of the 
guidance errors in the estimated state, P, does not imply that P is a cor- 
rect representation of the existing uncertainties. In fact, as the precision 
is reduced, the uncertainties indicated by the P matrix become an increas- 
ingly conservative representation of those that actually exist. This is due 
to the failure of the error propagation model to account for the increased 


errors in the estimated state that occur during the 


transition period between 


observations and velocity corrections. When no corrections are made to 
account for the increased errors, as in this study, it is shown that the opti- 
mal filter will give too low a weight to the information gained from space- 
angle observations. The resulting errors in the estimate of the vehicle* s 
state may, for large precision reductions, accumulate until they are exces- 
sively large. Velocity corrections are more poorly done as the precision is 
reduced because they are dependent on the estimate of the vehicle *s state and 
on a prediction matrix which projects the errors at any time into errors at 
the final time. This is shown to be due, at least in part, to the degradation 
of the prediction matrix which, in the lower precision cases, was affected to 
the extent that the determinant of one of its submatrices became negative near 
the end of the trajectory. 



INTRODUCTION 


References 1 and 2 present the results of a digital computer simulation 
of the midcourse phase of a eircumlunar navigation study. The Kalnan- Schmidt 
optimal filter (refs. 2, 3> and 4) is used to process simulated pilot -observed 
data (derived from the space geometry angles relating the vehicle to the earth 
and the moon) in order to produce an optimal estimate of the vehicle's posi- 
tion and velocity. With this information and a transition matrix relating 
end-point deviations to present deviations and knowledge of the state of a 
reference trajectory, the system computes and makes velocity corrections in 
the attempt to guide to a fixed end point at a given time . 

In certain situations it is desirable to simulate all or part of the 
midcourse navigation on a digital conputer which uses words of less precision 
than the 27 binary bits used in the studies reported in references 1 and 2. 

For an intelligent choice of word length, it is necessary to know the degrada- 
tion of performance due to decreased accuracy. It is the aim of this report 
to provide this information and at the same time to determine whether the 
original studies were done with words of sufficient precision. 


SYMBOLS 


a 

A 

A 1' A 2'[ 
A 3'\ J 

F 

H 

I 

K 

W[d,a 2 ] 

P 

P* 


column matrix of deviations in observed angles 

prediction matrix: a transition matrix relating deviations at the 

end point to present deviations 

3x3 submatrices of A 

matrix of partial derivatives of the equations of motion with respect 
to the state variables 

matrix relating observed angles and vehicle state 
identity matrix 
weighting matrix 

normally distributed with mean |i and variance a 2 
covariance matrix of estimation error vector, E[xx^] 
value of P after an observation 


p ,p f 
3' 4 J 


3x3 submatrices of P 
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X,Y,Z 
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A 
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observational error vector 

covariance matrix of the observational errors, E[qq T ] 

position deviation from reference 

error in estimate of r 

estimate of r 

root mean square 

position vector (subscripts indicate origin and end) 

magnitude of R 

earth radius 

moon radius 

general time arguments 

end-point time 

time of the kth observation 
velocity deviation from reference 
estimate of v 
error in estimate of v 

rms uncertainty of indicated velocity correction 
velocity-vector increment to be gained 
magnitude of AVq. 

six vector of position and velocity deviation from a reference 
trajectory 

estimate of x 

error in estimate of x, x-x 

space position coordinates 

observation of space angles, Hx + q 

estimate of y; Hx 



a 


declination of observed body 


(3 right ascension of observed body 

7 one-half subtended angle of observed body 

A increment 

a standard deviation of subscript random variable 

$(t 2 ;t 1 ) transition matrix relating state at t 2 to state at t x 

Notation Conventions 


n 

first-time derivative of ( ) 

( ) T 

transpose of matrix ( ) 

( r 1 

inverse of matrix ( ) 

E[ ] 

expected value of [ ] 

trace [ ] 

sum of the diagonal elements of [ ] 


Subscripts 

e 

end point 

E 

earth 

k 

at the kth observation 

M 

moon 

S 

sun 

V 

vehicle 


COMPUTER PROGRAM FOR SIMULATION 
General Description 

The digital computer program used in this study is a modified version of 
the FORTRAN II program for the midcourse navigation studies reported in 
references 1 and 2. 

The reference mission assumed for this study is the outbound leg of an 
earth-moon trajectory requiring about 78 hours for completion. Imperfect 
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injection conditions for the space vehicle are assumed, and it is the function 
of the navigation and guidance system to cause the space vehicle to arrive at 
the reference end point with a minimum of position error. To determine posi- 
tion and velocity the navigation system will process observations of the earth 
and moon made from the vehicle at preselected times. As a result of the knowl- 
edge gained from the observations, the system will generate an estimate of 
the state (position and velocity). On the basis of this estimate, the guid- 
ance system will make imperfect corrections to the vehicle velocity in an 
attempt to cause the vehicle to arrive at the reference end point (perilune). 

Twenty- seven second-order ordinary differential equations comprising 
three trajectories and six sets of three perturbation equations are integrated 
simultaneously. The integration subprogram utilizes a fourth-order Runge- 
Kutta method for starting and a Cowell "second- sum" method based on sixth dif- 
ferences for continuation. Double precision is used internally to control 
round-off error. 

The three trajectories are (l) the previously determined reference tra- 
jectory, (2) the actual trajectory giving the true state of the space vehicle, 
and (3) the estimated trajectory giving the current estimate of the vehicle 
state. Comparison of these three trajectories allows evaluation of changes in 
the system performance resulting from the reduction of precision in certain 
specified areas of computation. 


Program Areas in Variable Precision 

In order to study the effects of reduced precision in the various 
portions of the computations, five masks were developed which allowed indepen- 
dent control of the precision of each of the portions. 

The five levels of precision maintained with the masking technique are 
in addition to the normal 27-bit precision of the IBM 709^ • Mask 1 is used in 
conjunction with the optimal filter and its weighting matrix. Mask 2 is used 
to control the precision of the estimated trajectory. Mask 3 controls the 
precision of the space angles computed from the estimated state and the space- 
angle geometry. Mask 4 controls the precision of the matrix F, the perturba- 
tion equations, and the transition matrix, 4. Mask 5 controls the precision 
with which the prediction matrix. A, is updated, the precision of the guidance 
equation computations , and the computations involved in the velocity correc- 
tion model used to implement a velocity correction. Full 27-bit precision is 
used on those calculations that provide information about the performance of 
the system and on calculations involved with the actual and reference trajec- 
tories. In addition, the numerical integration subroutine used to integrate 
the equations of motion and the perturbation equations carried out internal 
computations in 54 binary bit precision (double precision) as mentioned before. 
This was true regardless of the effective mantissa length of the second deriv- 
atives computed from the equations of motion and the perturbation equations or 
the initial conditions. A discussion of the masking operations used to reduce 
the effective word length is given in appendix A. 
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Processing an Observation 


The digital simulation of the complete system is represented in figure 1, 
which also shows in dashed boxes those areas which are maintained at the pre- 
cisions specified by masks 1, 2, 3 j> and 5* A description of the operation 
of the system and the equations used is given in the remainder of the section. 
The circled numbers in the text refer to corresponding numbers in figure 1 and 
are used to show which of the boxes is being discussed at the moment. 

Updating the trajectory.- When an observation has been made at time t^. 
and is to be processed, numerical integration of the equations of motion is 
initiated (l) for the actual, estimated, and reference trajectories simulta- 
neously. Initial conditions for the actual and reference trajectories are 
merely the states which existed at the time of the last observation, while the 
estimated trajectory uses the new estimated state derived from processing the 
last observation. Also, simultaneously with (l) the sixfold perturbation 
equations (g) are integrated numerically from the time of the last observation, 
^k i* to P rod - uce a transition matrix (3) . This process continues until the 

computer time equals the time of the observation, t^. The integration is then 
stopped and the covariance matrices are updated. 

Updating of the covariance matrices P(t^) and Q(t-^).- The updating opera- 
tion on P(t^) is accomplished through the use of linear prediction. This 
method requires the assumption that small deviations of state at time t^ can 
be obtained from a linear combination of the deviations at time t-^^. i n par- 
ticular, when dealing with the deviation x(t^_ 1 ), the relationship may be 
written in matrix form as follows: 

x(t k ) = $(t k ; t k _ 1 )x( t k _ 1 ) (l) 


where the transition matrix from time t^^ to time t^. Sub- 

stituting equation (l) into the definition 


P(t k ) = E[x(t k )x T (t k ) ] 


( 2 ) 


yields the following updating equation for PCt^^: 


P(t k ) = ®(t k jt k . 1 )P(t k _ 1 )® T (t k ;t k _ 1 ) (3) 

The covariance matrix of the observational errors, Q(t k ) , is given by 

~o z 0 0 

Q(t k ) = 0 a 2 0 | (4) 

0 0 a : 
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where a 2 = 100 + (O.OOI 7) 2 seconds of arc and 7 is l/2 the subtended angle 
of the observed body. 

Thus the updating operations at © consist of computing equation ( 3 ) 
and forming Q(t) . 

Determination of the weighting matrix K(t k ) The next step is to com- 
pute the H matrix followed by the weighting matrix K(t k ) . The matrix H(t k ) 
is computed © from the reference state values in the form 

Ba Ba 

By Bz 

Bp Bp 
By Bz 

By By 

By Bz 

where the partial derivatives for the body being observed are given in table I. 
The matrix K(t^) is then computed © from 

K(t k ) = P(t k )H T (t k )[H(t k )P(t k )H T (t k ) +Q(t k )]" 1 (6) 

With K(t k ) known, the P‘(t k ) matrix is then computed © from 

P‘(t k ) = P(t k ) - KttpHCtpPCtp (7) 

reflecting the change in P due to the observation just processed; 1 P*(t k ) is 
then stored. The delay unit © represents the storage of P'(t k ) until the 
time of the next observation. 

Two quantities are derived from P’ and are used to assess the system*s 
estimation performance. These two quantities, r rms and v rmg , are the standard 
deviations of the error in estimating the magnitude of the position and veloc- 
ity deviations from the reference. They are defined by 

r = [trace (P ’)] ^ (8) 

rms ' 1 

and 

^rms = t trace ( p 4 ’)f /2 (9) 

1 The order of matrix multiplication is from left to right. It has been 
found that even 27 bits is not sufficient to preserve the non-negative definite 
property of the P matrix if the order of multiplication is not preserved. 
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where P 1 ’ and P 4 ' are 3 x 3 submatrices of the P* matrix partitioned as 
follows : 


P* 




( 10 ) 


Formation of the new estimated state .- In this system space-angle obser- 
vations consist of the measurement of three angles related to the body (earth 
or moon) under observation. These angles are a, the right ascension; 3, the 
declination; and y , if 2 of the angle subtended by the body. 

The estimated angles are computed (§) from the space geometry equations 
(table II ) using the state of the estimated trajectory. From the same space 
geometry equations, and using the state of the actual trajectory, the actual 
angles are computed (1(5) . The difference between these two sets of angles is 
found and noise, scaled according to Q,(t^) , is added. The result © multi- 
plied by K(tjj) ^2) is added to the estimated state (l^) to produce a new 
estimated state. This completes the estimation process. 

Computation of the indicated velocity correction .- In order to provide 
in-flight velocity correction capability, a prediction matrix, A(t e ;t^), which 

predicts the end-point state from the present state is computed (3) from 


A (t^;t v ) - A(t^;t^_ i )$(t v ;t v _ n ) 


- 1 


k J k-i 


( 11 ) 


Since the initial prediction matrix, A(t e ;t 0 ), is an input quantity, it is only 
necessary to update the matrix at each observation. This requires finding 

from ® ( Wi ) - 

Because a syraplectic matrix (ref. 5 ) > it is invertable 

merely by rearranging terms and changing signs. Thus, if ^(t^jt-^. x ) is par- 
titioned into 3 x 3 submatrices, then 


t ( t k it k-i) “ 

and 

1 = 


Therefore, there are no errors in this inversion process. 


*1 *2 
*3 *4 


T T 

$ -4 1 

*4 *2 


-®. 


( 12 ) 


( 13 ) 
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With the difference between the estimated and reference states, r and v, 
(l^) and the updated prediction matrix from equation (ll), the indicated veloc- 
ity correction, AVq, is computed Q.^) from 


av g 


A ' 1 A 
2 1 


[x] 


( 14 ) 


The magnitude of AVq is 


l AV ol 


(AV g T AV g ) 


1/2 


(15) 


This concludes the computation cycle done at each observation and the 
computer is now ready to process the next observation unless a velocity correc- 
tion is to be executed. In this case the correction is made after a time delay 
in order to simulate the time required to orient the vehicle along the desired 
thrust vector. To accomplish this delay, processes (l) , (g) , and @ are 
repeated until the computer time equals the time at which the velocity correc- 
tion is to be made. At this time processes © , (lj) , Q^) > and QlJ) are 
repeated and computation control is passed to a separate program section for 
the performance of the velocity correction maneuver. 


Performing the Velocity Correction Maneuver 

The flow lines representing velocity correction increments are shown 
dashed in figure 1 to emphasize the occasional nature of the velocity correc- 
tions. Also, these corrections are assumed to occur instantaneously so that 
no position or time change need be accounted for during the maneuver. 


Correction of the actual and estimated states .- The inputs to the veloc- 
ity correction model are AVq and the random execution errors which consist 
of cutoff error and pointing error. The cutoff error for this study is 
assumed to be random with normal distribution N[0, (o^y) ] where oxy = 1 per- 
cent. The pointing error is resolved into errors in right ascension and 
declination. Both of these angles are assumed random with distribution 
N[0,o 2 ] where a = 1°. The output ^j) of the velocity correction model is 
the correction applied to the actual state. The correction to the estimated 
state is the measured value of that applied to actual state . The errors in 
the measuring device are assumed to be random with distribution 
N[0,(l cm/sec) 2 ] in each axis. These errors are added to the actual values at 
(©) to simulate the estimated values. The covariance matrix of these errors 
in measuring the correction is added to the P matrix (^) as follows: 

["(l cm/sec) 2 0 0 1 


P = P ’ + 

r 4 4 


0 

0 


(l cm/sec)‘ 
0 


(l cm/sec)' 


(16) 
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where P 1 is the value of P 4 before the velocity is corrected. This com.' 
pletes the velocity correction maneuver. The computer now continues on to 
process the next space- angle observation. 


Generation of Initial Prediction Matrices 

To provide the initial prediction matrix A(t e ;t^) in equation (ll) the 
system was, prior to the actual run, commanded to run from injection to the 
end point without observations or velocity corrections. This resulted in a 
transition matrix $(t e ;t Q ), which was the initial value of A(t e ;t Q ) desired 
as an input for the actual run. A different A(t e ;t 0 ) was generated in this 
way for each word length to be used. 

RESULTS AM) DISCUSSION 


In this section the results of 
cision are presented and discussed. 


1. 

The 

effect 

of 

word 

length 

2. 

The 

effect 

of 

word 

length 

3- 

The 

effect 

of 

word 

length 

k. 

The 

effect 

of 

word 

length 

5- 

The 

effect 

of 

word 

length 


the effects of reduced computational pre- 
The areas to be considered are: 

on earth-moon trajectories. 

on the optimal filter. 

on the error propagation model. 

on state estimation. 

on guidance . 


Effect of Word Length on Earth-Moon Trajectories 

In order to determine the effect of word length on trajectories from the 
earth to the moon, the aperture in mask 2 was set to the desired bit length 
while all other masks were set to 27 bits. The equations of motion for all 
three trajectories were then numerically integrated simultaneously from injec- 
tion to the end point at reference perilune without making any observations. 

When the estimated and reference trajectories were compared at the time 
of reference perilune, t e , the range magnitude between the two end points, 
shown in figure 2, gave information showing how position error increased when 
the precision was reduced. This curve shows that, as precision is reduced 
below about 25 bits, the error that accumulates over the entire trajectory 
rises sharply. This tends to limit acceptable word lengths to the range from 
25 to 27 bits. However, reference to figure 2 shows that a 25 “bit reference 
trajectory, for example, would be in error by l8 km which would be unsatisfac- 
tory in most cases. This indicates that 26 or perhaps 27 bits would be 
required for reference trajectories. On the other hand, estimated trajec- 
tories accumulate errors involving the numerical integration of the equations 
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of motion only between observations and velocity corrections, and this study 
has shown that satisfactory results may be expected with precisions as low as 
21 bits. 


Effect of Word Length on the Optimal Filter 

The behavior of the covariance matrix P as a function of word length 
is used here as a means of assessing the degradation in the performance of the 
filter as characterized by equations (3) through (7)« Errors in those calcula- 
tions involved in producing the updated P matrix as the result of making an 
observation, that is, operations (2) , © , © , ©, © , and © in figure 1, 

will be detected by f rms and v rms defined in equations (8) and (9) . Error 
detection will consist of two steps: first, the effect of the transition 
matrix on the updating of P(t k ) from time t^ to t k in equation (3), and 
second, the effect on P^t^-) in equation (7) will be shown. 

Effect of transition matrix on the updating of P(t k ).- In table III the 
quantities ^ rms for both l6 and 27 bits are given just prior to each sequence 

of observations and after each velocity correction. Relatively long periods of 
time elapse between injection and the beginning of the first sequence of obser- 
vations and between each velocity correction and the succeeding observations 
sequence. If the transition matrices for these time periods were erroneous for 
l6 bits it would be expected that r rms and v Tms for l6 bits would differ con- 
siderably from its 27-bit counterpart at the beginning of the next observation 

sequence. Referring to table III and comparing r and v for l6 and 27 

rms rms 

bits in each row of the table before a velocity correction and at the beginning 
of the next observation sequence shows clearly that transition matrix effects 
are quite small. 

Effect of precision on P*(t k ).- With the computation of P*(t k ) in equa- 
tion (7) the optimal filter calculations are complete. If there were signifi- 
cant errors due to reduced precision in the calculations leading up to 
computation of P T (t k ), it would be reasonable to expect that a series of 
observations taken close together would lead to serious errors in P*(t k ). The 
results of four sequences of observations in which the observations were taken 

close together is given in table III. However, when r and v are com- 

-L in rms 

pared at the beginning of an observation sequence and again at the velocity 
correction following that sequence, it is seen that these quantities are not 
adversely affected by the reduction of precision from 27 bits to l6 bits. It 
is, therefore, concluded that precisions as low as l6 bits would provide 
adequate results in the filter computations. 


Effect of Word Length on the Error Propagation Model 

The performance of the system in estimating the errors in state is 
heavily dependent on the ability to correctly propagate errors at one time into 
corresponding errors at a later time. This operation is expressed 
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mathematically in equation (l). Since equation (3) is derived directly from 
equation (l), it can be seen that the system will not "know" the true error in 
estimate if ®(tj s; ;tj c _ ) does not correctly represent the way in which errors 
propagate from one time to the next. 

Figure 2 shows that large deviations of the estimated trajectory from the 
reference trajectory develop for the larger reductions in precisions. Yet, 
from table III it was shown that the P matrix was virtually invariant with 
precision. Therefore, tjj_ - l) is at fault and does not represent the true 

way in which errors propagate. Since 

^ ) ( t k-' t k- 1) = \ ^(t) < l , (t k ;t k )dt (17) 

Jt k-i 

and since the numerical integration subroutine has high internal precision it 
is obvious that any difficulties with $(t k ;t k _ x ) stems from F(t). The end 
result of using the transition matrix derived from equation (l7) is shown in 
figure 3* Here, r, the true error in estimate is compared to r rms for the 
case where all the masks had a l6-bit aperture. It is seen that r rms and, 
therefore, P, is much too conservative to adequately represent in a statistical 
sense the errors in estimate which actually existed. As a result, the system 
believes that its knowledge of the estimated state is better than it really is. 
This causes the optimal filter to pay less attention to observational informa- 
tion than it should. 

Appendix B gives a method for overcoming the difficulties with F(t) , but 
its practical value is limited by the requirement of detailed knowledge of the 
way in which the individual errors, caused by reduced precision, propagate with 
time. For this reason it was not used in this study but is presented as a 
method that might be applicable to a less complex problem or in the event 
future study should reveal a simple model of the propagation of the reduced 
precision errors. 

The question of how F(t) is affected by reduced precision is answered by 
comparing $(t e ;t Q ) for 16 bits with the transition matrix for 27 bits. This 
comparison shows that some degradation accumulates when t e and t Q are sepa- 
rated in time by as much as 12 hours, but for shorter periods of an hour or 
less 0(t e ;t o ) is essentially invariant with precision. Therefore, if long 
transition periods are to be encountered, precisions higher than l6 bits are 
required in the calculation of F(t). 


Effect of Word Length on State Estimation 
Through Space- Angle Observations 

Navigation with the system being investigated requires observations of 
one or more space angles which are related to the vehicle state through the 
space- angle geometry. When an observer measures these space angles with an 
imperfect instrument, the quality of the knowledge gained will depend on the 
combined instrument and observer accuracies. These actual space angles must 
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then he fed into a computer. The precision with which the angles are accepted 
by the computer is dependent on its word length. In this simulation, the input 
precision is controlled by mask 3* 

Effect of the space angles .- Two computer runs were made in which mask 3 
was 27 bits in one and 16 bits in the other. All other masks remained fixed at 
l6 bits. The similarity of these runs was such that the maximum position 
deviation between the two estimated trajectories was approximately 11 km. This 
indicates that bits of lower significance than 16 in the space angles tend to 
be ignored in the updating of the estimated state. This implies that little or 
no improvement in the estimated state can be expected from increasing the pre- 
cision of the space angles unless the precision of K(tjj.) is also increased. 
This loss of the lower order bits results in a considerable loss of system sen- 
sitivity to position errors since the quantization error in the l6th bit is 
quite large in terms of seconds of arc. 

Effect of the P matrix .- The effect of a conservative ?(%) matrix on 
the correction of the estimated state at in figure 1 may be seen from the 

following simplified example: Suppose the state vector is a 1 X 1 and only one 
variable is being observed. Then all the matrices are 1X1. Suppose also 
that the output vector at (ll) in figure 1 is the column matrix a. Then the 
correction to the estimated state at (l|) is K(tj^)a which, from equation (6) 
using 1X1 matrices and dropping the subscript, t^, is 


Ka = 


PH „ 

— 2 a 

PH + Q 


1 P 

H p + q/h 2 


(18) 


At time t^ both H and Q, are constants. Sketch (a) shows how K, and there- 
fore, the correction to the estimated state increases with increasing P. 



Sketch (a) 


Therefore, a P matrix which is too 
conservative will always result in too 
small a correction to the estimated 
state even if the actual space angles 
are sufficiently accurate. 

Errors in correcting the esti- 
mated state .- When the P matrix is a 
very conservative estimator of the 
existing errors and the K matrix 
causes large numbers of the least sig- 
nificant bits from the space angles to 
be ignored, errors in the estimated 
state may accumulate rapidly with each 
additional observation. This situation 
is shown in figure ^ where very large 
peaks develop in the midcourse region 
where the rate of change of the space 
angles is low. 
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Figure 5 is a plot of r for the l6-bit case shown in figure 4. 

Included are points before and after each observation rather than only after 
each observation as in previous plots. The growth of errors between observa- 
tion sequences is also shown. Inspection of figure 5 shows that the effect of 
some of the observations is to actually make the estimated state worse. When 
observations are switched from one body to another in this region, the space - 
angle differences are large because of the large deviation between the actual 
and estimated state. The first observation of the moon is this sequence 
(observation 20) was made with a most dramatic drop in r. To examine why a 
moon observation was so effective, an error ellipsoid is used. 

When the P matrix is partitioned into four 3X3 submatrices as in 
equation (lO), the submatrix P 1 may be represented as a three-dimensional 
error ellipsoid of position and, similarly, P 4 an error ellipsoid of velocity. 
The relative magnitude and direction of the axes represent the relative magni- 
tudes and directions of the uncertainty at the particular vehicle position in 
space . 


Changes to the error ellipsoid, as the result of an observation, are 
indicative of the knowledge gained from the observation. In particular, these 
changes are twofold. First, except when near the observed body, the major 
axis of the ellipsoid tends to aline itself with the vector to the observed 
body* Second, a reduction in the ellipsoid volume occurs since, by equa- 
tion (t), an observation requires that the P matrix must be reduced or left 
unchanged. Also, a large reduction in the volume of an ellipsoid implies a 
large correction to the estimated state. 

Figure 6 shows the position error ellipsoid both before and after obser- 
vation 20. Included are vectors indicating directions from the vehicle to 
both the earth and the moon. Because of the immediately preceding four earth 
observations, the major axis is only 10° from the vehicle-earth vector and the 
ellipsoid is quite elongated. As a result of the moon observation, the elonga- 
tion is considerably reduced and the volume is reduced by a factor of 2.46. 

The orientation of the ellipsoid is skewed toward the vehicle-moon vector. 

In summary, the computation of the space angles with precisions greater 
than that employed by the K matrix will not improve the ability to correct 
the estimated state by a significant amount. Furthermore, the effect of a P 
matrix which is too conservative an estimator of the existing errors is to 
cause the K matrix to give too low a weight to space-angle information. This 
results in corrections to the estimated state which are too small at each 
observation. The conservatism of the P matrix may be remedied by correcting 
the error propagation model or by augmenting the filter computation as 
outlined in appendix B. 


Effect of Word Length on Guidance 

The evaluation of the effect of precision on guidance was made by con- 
sidering the effects on the indicated velocity correction vector, AVq, due to 
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the estimated state and the prediction submatrix, A 2 . Also considered is the 
way in which reduced precision affects the actual state through the velocity 
corrections. 

Effect of precision on AVq.- Figure 7 shows the magnitude of AVq at 

each observation and velocity correction for l6, l8, and 27 bits. The separa- 
tion of the curves for the three different precisions at the time of the first 
velocity correction (12 hours) is shown to differ from the 27-bit value. The 
separation at this time increases rapidly as the precision is reduced and is 
to be expected because of the dependence of AVq on x given in equation (l4). 
The separation of the three curves at subsequent velocity corrections should 
be expected to increase because immediately after the first velocity correc- 
tion, if it were perfect, AVq would be zero. The execution of AVq was, 
however , not perfect but at least the pointing and cutoff errors in each case 
were the same except for the number of least significant bits discarded by the 
masking. Therefore, the growth in the magnitude of AVq in the time period 
between a velocity correction and the next observation is due to the errors in 
determining x and the errors in executing the velocity correction. 

It is interesting to note that the erratic behavior of |AVq| is rapidly 
reduced with time until about 7^*5 hours in the l8-bit case. At this point, 
a sharp dip began and was followed a short time later by a second even sharper 
dip. Since the computation of AVq depends on the behavior of A" 1 , a plot 
of the determinant of A 2 was made and is shown in figure 8. Clearly, the 
peculiar behavior of the l8-bit case with an l8-bit prediction matrix was 
associated with a large dip during which the determinant of A 2 was negative 
for a short while. Also shown is the unusual behavior of the l6-bit case when 
a l6-bit prediction matrix was used. Here, the curve becomes negative some- 
time after 38 hours and remains so throughout the remainder of the run. 

The peculiar behavior of the determinant of A 2 may be explained as fol- 
lows: If infinite precision were used, the product of all the individual 

transition matrices between observations and velocity corrections would be 
given by 


8 *(t 1 ;t 1 _ 1 )tt 0 ;t 0 ) = »(t e ;t 0 ) = A(t e ;t 0 ) ( 19 ) 

i=i 

where n < t(tj_;t^_ 1 ) < I > (t 0 ;t 0 ) means 
i= 1 

$(t e ;t e _ 1 ) . . . ®(t 2 ;t J«(tjjt 0 )®(t 0 ;t 0 ) (20) 

and 

$ (t 0 ;t 0 ) = I 

It is obvious that at the end point the prediction matrix under these ideal 
conditions is given by 
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A(t e jt e ) - 


( 21 ) 



If A x , A , A 3 , and A 4 are 3 X 3 submatrices, then A = A 4 = I and 

A 2 = A 3 = 0. In general, however, the left side of equation ( 19 ) does not 
equal A(t e ;t 0 ). This is because the product of individual piecewise transi- 
tion matrices which have been generated from exact initial conditions by numer- 
ical integration over successive short-time intervals yields a different final 
result than a transition matrix integrated from exact initial conditions over 
a time period which is the sum of the successive short-time intervals. There- 
fore, at the end point A 1 and A 4 do not equal I and also A 2 and A 3 do not 

equal zero because of accumulated errors. As the trajectory nears the end 
point, these errors dominate the situation more and more with the dominance 
occurring earlier as the precision is reduced. This can be seen by comparison 
of the l6- and l8-bit cases in figure 8. The effect of substituting an initial 
A matrix generated with 27-bit precision may also be seen by comparing the 
16 -bit case using a 27-bit initial prediction matrix and the 27-bit case using 
a 27-bit A matrix with the l6-bit case results in a curve which is quite 
similar to the 27-bit case in which a 27-bit starting A matrix is used. 

(Note that determinant of A 2 is not zero at the end point as would be 

expected if infinite precision had been used.) 

The effect on guidance due to the use of a 16- and 27-bit A matrix is 
given in table IV. Here it is seen that the results with the 27-bit A matrix 
are generally better. Furthermore, at the end point the results are markedly 
different as would be expected because of the negative determinant of A 2 at 
the time of the third correction. 

Effect of guidance errors on the actual state .- Although the actual tra- 
jectory was confuted in full precision, it is affected by the estimated state 
through velocity corrections. Table V shows in column 1 the standard 27-bit 
case. In column 2 is shown the effect of reducing the precision of the esti- 
mated trajectory to 16 bits (mask 2) when all other masks were set to 27 bits. 
Column 3 shows the effect when all masks have 16 -bit apertures except for 
mask 2 which was 27 bits. Columns 4 and 5 show the cases when all masks had 
16 -bit apertures but different starting prediction (A) matrices. All starting 
prediction matrices were 27 bit except for the case shown in column 5 where a 
16-bit starting prediction matrix was used. In all columns the magnitude of 
r, |r|, is given along with |r|, the magnitude of r. The quantity |r| 
is included because jr| depends on the previous value of r. Referring to 
equation (l4) we see that the results in column 2 show the effect of x on the 
actual state and on the estimated state. Similarly, column 3 shows the effect 
of reduced precision on the computation of A 2 1 A 1 . Column 4 shows the combined 

effects of precision on x and on A^A^ Column 5 shows, in addition to that 

shown in column 4, the additional effect on A~ 1 A 1 of a l6-bit starting 
prediction matrix. 
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The end-point results for columns 2, b, and 5 are obviously unsatisfac- 
tory from a guidance point of view. Column 3 is somewhat questionable and in 
general would be considered unsatisfactory. To improve the results in column 
3 would require greater precision in forming A” 1 A 1 . 

In summary, the prediction matrices generated with precisions of l8 bits 
and lower should not be used. In fact , a 27-bit prediction matrix should 
always be used if available. Confutation of A" 1 A 1 in the guidance equation 
should be done with the aperture in mask 5 no less than 20 bits. 


CONCLUSIONS 


The results of this study show that satisfactory performance of the 
navigation and guidance scheme proposed in NASA TN D -1208 and MSA TR R-135 
can be acconplished at greatly reduced precision in some general areas and 
with lesser reductions in others provided that the increased errors are prop- 
erly accounted for. This should allow similar simulations to be carried out 
by computers using greater precision only in certain critical areas rather 
than throughout the entire simulation as would have been required heretofore. 
The general areas and the corresponding precisions required are as follows: 

1. The precision of the estimated trajectory must be about 21 to 27 
bits (mask 2) . 

2. Reference trajectories require a precision of 26 or 27 bits. 

3. The confutations embodied in the optimal filter may be confuted with 
precisions as low as 16 bits (mask l) . 

The transition matrix ^('ti c ;tj 5 ._ 1 ) (controlled by mask may be 
derived from F(t) where 16 -bit precision is used. However, if t-^ and t^ 
are widely separated in time, the precision should be increased. For 
^(t^;tj c _ 1 ) to represent the correct error propagation model requires that 

F(t) and/or the filter confutations be augmented to take into account the 
increase in uncertainty. 

5. Increasing the precision of the space-angle confutations (mask 3) 
over that emp loyed by the weighting matrix, K, was found to produce only a 
negligible improvement in the estimated state. 

6. The effect of a P matrix which is too conservative an estimator of 
the errors in the estimated state is to cause the weighting matrix, K, to give 
too little weight to space-angle information. This can contribute to the 
build-up of very large errors in the estimated state. 

7. Prediction matrices generated with precisions of l8 bits and lower 
should not be used. If available, 27-bit prediction matrices should be used. 

The confutation of A 2 1 A 1 in the guidance should be done with the aperture in 
mask 5 greater than about 20 bits. 
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o. The above conclusions lead to the further conclusion that the 
previous studies, which used 27-bits precision throughout, were done with ade 
quate precision. 

Ames Research Center 

National Aeronautics and Space Administration 
Moffett Field, Calif., Jan. 10, 19 66 



APPENDIX A 


WORD LENGTH REDUCTION 


In FORTRAN II, floating-point arithmetic is used throughout (except for 
indexing and bookkeeping) to maintain the decimal point of numbers automati- 
cally. Internally the IBM 'JO^h uses floating-point binary numbers which are 
presented as a signed proper fraction, called the mantissa, times some inte- 
gral power of 2, called the characteristic or exponent. The number or com- 
puter word is 36 binary bits, 1 made up, from left to right, of a sign of the 
mantissa, an 8-bit characteristic, and a 27-bit mantissa. The precision with 
which a number may be represented may be varied by changing the number of 
binary bits in the mantissa where 27 is the maximum. 

The function of the mask is to specify in the Boolean "and" operation the 
number of successive least significant bits of the mantissa to be set to zero. 
This is done by placing in the corresponding bit of the mask a "l" where a bit 
is to be retained and a "0" where a bit is to be discarded. The following 
sketch illustrates a masking operation in which a floating-point number, A, is 
to be truncated from 27 to 23 bits. If in the sketch the mask is the number, 

B, and the truncated result is C, then the masking operation may be described 
by the Boolean expression 


A*B = C 


where the "*" is the "and" operator. 


A 

B 

C 



Thus, the "and" operation places bit by bit a "l" in C if both A and 
B contain a "l," otherwise C is set to "0." The net effect, then, is to 
retain the bits of A where the corresponding bit in B is a "l." 


•‘■Reference Manual IBM 709^ Data Processing System. 
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APPENDIX B 


CORRECTION OF P MATRIX TO COMPENSATE FOR INCORRECT 
ERROR PROPAGATION MODEL 

In order to correct the P matrix to compensate for incorrect error 
propagation model, suppose the equations of motion for deviations from the 
reference in the 27-bit case are given by the linear differential equation 


where 


x = Fx 
Sx 

& y 

Sz 

5x 

by 

Sz 


(Bl) 


(B2) 


and the 6 ? s represent deviations from the reference. Now , in reduced pre- 
cision, it is assumed that the equations of motion have a term missing which 
may be accounted for by the addition of the term FpX^ so that, for the 

l6-bit case which will be used hereafter as an example, equation (Bl) becomes 

^27 “ ^16 X 16 + ^P X P (®3) 


Here is some state vector which is not updated and is active only from 

one observation to the next; Fp is a matrix whose elements are assumed to be 
functions of time and the precision of the integration of the equations of 
motion. The transition matrices $ 13 and ^ 16 are found from: 


® 16 (tito) - 

V. 

/ F (x)®(T 0 )d T 

(B4) 

x o 



pt 

(B5) 

/ F (TH(-r 0 )dT 
J t„ p 


o 


where $(t 0 ) and represent the proper initial conditions. 
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The deviations at some later time, t, can now he found from the 
expressions 


and 


• 16 (t) = 4 16 (tjt 0 )x(t 0 ) 

(b6) 

Xp(t) = * 16 (t;t 0 )xp(t o ) 

(B7) 


Let x be the vector sum of x and x where x is the estimate of x, and 
x is the error in the estimate of x. Thus, 


* 16 ^ = X 16 (t) + X 16 (t) 


x 27 (t) = x 27 (t) + x 27 (t) 


(B8a) 

(B8b) 


To simplify writing the following equations, let 

he * he^o) 


(B9a) 

(B9b) 


Sketch (b) is a vector diagram showing the actual, estimated, and refer- 
ence states at times t Q and t, 




Act (to) 


The state at t is the result of numerical 

integration of the equations of motions 
from t Q to t using the initial con- 
ditions at t Q . When 27-bit precision 
is used, the estimated state at t is 
shown as the point Est 27 (t) . The 
estimate of x 27 (t) , x 27 (t), is the 
vector from the reference state, Ref(t), 
to Est 27 (t) . The error in the esti- 
mate, x 27 (t), is the vector from 
Est 27 (t) to Act 27 (t) in accordance with 
equation (B8b) • On the other hand, 
when l6-bit precision is used, the 
estimated state at t is shown as the 
point Est 16 (t). In this case, applica- 
tion of equation (B8a) does not yield 
X 27 (t)> the true state at t. Instead, 
it yields the state at Act 16 (t) which 
is in error by Xp(t) because of the inability of equation (b 6) to produce the 
true error vector, x c (t) which is given by 



Est(to) 


Ref (to) 


Act 27 (0 


X c (t) — X 16 (t) + Xp(t) 

= $ 16 X 16 (t o) + t 16 Xp(t 0 ) 


(Bio) 
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and, as seen from sketch (b), is related to the true state by 


x 27 (t) = x 16 (t) + x c (t) (Bll) 


The true covariance matrix of the error in estimate is defined as 



P c (t) = E[x c x c T ] 

(B12) 

When (BIO) 

is substituted into (B12) 


P c (t) 

= ^*16^ + Xp(t)][x^ 6 (t) + Xp T (t)]j- 



= E[x 16 (t)x^ 6 (t) + x ls (t)xp T (t) + Xp(t)x^ 6 (t) + x p (t)xp T (t)] 
= 0 16 E[X 16( t o) X l 6 ( t o)^i 6 + ^ 16 E ^ 16 (t 0 )Xp(t 0 ) T ]^ s 



+ t i6 E[Xp(t 0 )xT 6 (t 0 )]^ 6 + t is E[x p (t 0 )x p T (t 0 )]^ 6 

(B13) 

Let 

Pi 6 ( t 0 ) = E[x is (t 0 )x^ s (t 0 )] 

(Bl4) 


P i6^) = *i6 P ie (t o)h 6 

(B15) 


C(t Q ) = E[x 16 (t 0 )xp T (t 0 )] 

(Bl6) 


D(t 0 ) = E[x p (t 0 )x p T (t 0 )] 

(B17) 


Wow (B13) becomes 

p c (t) = P l6 (t) + ♦ M 0(t o )t^ 8 + t 10 C(t o ) T ^ a + * M D(t 0 )V^ (B18) 


The matrix C gives the correlation of the vectors x 16 and Xp . Therefore, 

C must be updated in order to reflect the change in x due to its transla- 
tion in time from t Q to t. Thus, at time t, equation (Bl6) becomes 

c(t) = E[x c (t)xp T (t)] (B19) 
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Substituting for x c (t) from (BIO) , we have 


C(t) = E |[$ 16 x l6 (t 0 ) + i 16 ^p( t 0 )^(t o ) T ^ 6 j 


= E 


T m T m 

1 ,T , / 4- N ( 4- \ ,|,T 


*i6 X 16 (t O )x p (t O ) *16 + *i6 X p (t o) X P < t O ) * 


16 


= ® C(tJt T + t, D(tJf T 

16 ° 16 16 O' 16 


(B20) 


Returning to sketch (b) , the situation is shown where the three trajec- 
tories (reference, actual, estimated) have been integrated from t Q to t, but 
only k-1 observations have been made. When the kth observation is made at 
time t both P (t) from (Bl8) and C(t) from (B20) include information only 



With P c (t k _ 1 ) known, P c (t k ) is 


from the previous k-1 observations. 
Therefore, to avoid confusion, it is 
desirable to place an additional sub- 
script on t to indicate whether or 
not updating has taken place. Thus, 
P(t k _ 1 ) and C(t k ) must be updated to 

time t k to include the new knowledge 
gained from the kth observation. 

This new knowledge allows the esti- 
mated state to be corrected as shown 
in sketch (c). 

computed from the usual equation 


P c (t k ) = [I - K(t k )H(t k )]P c (t k _ 1 ) (B21) 


where 


K(t k ) = P c (t k _ 1 )H 1 '(t k )[H(t k )P c (t k . 1 )H x (t k ) +Q(t k )3 _1 (B22) 


To 


update 0(1^ ) to time note that from sketch (c) 

■^c^k-i) = x 27^k-i^ " x i6^k-i) 


and also 


x c^k^ “ x s7^k^ “ x i6^k^ 
C 27 


= " ^i6 (t k) 


(B23) 


(B24) 
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From reference the actual observed space angles, y, are given by 


y(t k ) = H ( t k) x 27 ( t k _ 1 ) + q( t k ) ( B2 5) 

where q(t k ) is the error in measuring the angles. The estimate of the same 
space angles, y, is given by 

y(t k ) = H(t k )x is (t k _ i ) (B26) 

The error in the estimate of the space angles is given by subtracting (B26) 
from (B25 ) . Thus , 

y(t k ) - y(t k ) = H ( t k )Cx 27 (t k _ i ) - x(t k _ i )] + q(t fc ) 

= H(t k )x c (t k _ i ) + q(t k ) (B27) 

The new estimate of the state at time t k is from reference 3 

x i 6 ( V = + K(t k )[H(t k )X c( t k -i ) + q(t k)] ( B28) 

Substituting (B28) into (B24) and using (B23) yields 

x c (t k ) = x c( t k - 1 ) " K(t k ) [H(t k )x c ( t k-i) + qC'bjj)] (B29) 

which is the equation for updating x c from time t k _ 1 to t k . Before pro- 
ceeding with the derivation of C(t k ), it is noted that q(t k ) is a vector 
such that 

Q(t k ) = E[q(t k )q T (t k )] (B30) 

where Q(tj c ) is the covariance matrix of the observational errors and is 
assumed to be a known function of time. Also, it will be assumed that q(t) 
and Xp(t) are uncorrelated which results in 

E[q(t k )x p T (t k _ i )] = 0 (B31) 

Now, to find C(t k ) equation (B29) is substituted into equation (B19) 
and using equation (B31) it is found that 


2b 


C(t k ) = E |x c (t k _ 1 )x p T (t k _ 1 ) - K(t k )[H(t k )x c (t k _ 1 ) + q(t k )]x| p T (t k _ 1 )| 

= E[x c (t k _ 1 )x p T (t k _ i )] - K(t k )H(t k )E[X c (t k _ i )x p T (t k-i )] 

- K(t k )E[q(t k )x p T (t k _ i )] 

= C(t k _ 1 ) - K(t k )H(t k )C(t k _ i ) (B32) 

To find the corrected P matrix, P c , where there is increased uncer- 
tainty due to increased error in the estimated state, the computational steps 
are summarized below: 

(1) To update in time from t Q to t, compute: 

c(t) = » ls C(t 0 )/ i6 + + 16 B(t 0 )^ 6 
P 0 (t) » P M (t) + » 1B C(tX s + * 16 C(t 0 )V s + ♦ ls D(t 0 )^ 

(2) When t is the time of an observation t^, then step (l) gives 
C(t k _ i ) and P(' t k _ 1 )« To update as a result of the observation compute: 


P c (t k ) = [I - K(t k )H(t k )]P c (t k _ 1 ) 
C(t k ) = [I - K(t k )H(t k )]C(t k _ i ) 


This method allows the P matrix to be corrected for errors due to 
reduced precision providing Fp and Xp in equation (B3) can be found. 

Unfortunately, this may not be easy even for relatively simple situations. 
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Rg radius of earth 

Hjyj radius of moon 

Rgj^ earth -moon distance 
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TABLE III . - COMPARISON OF r rms AND v rms FOR l6 AND 27 BITS 



16-bit word length 

27-hit word length 

Time j hr 

Comments 

r rms> 

km 

v rms' 

m/s 

r rms> 

km 

^rms^ 
m/ s 

3.5 

At time of first 
observation 

W.57 

4.820 

48.62 

4.83O 

12 

Immediately after 
executing first 
velocity correction 

16.82 

.533 

16.83 

• 533 

24 

At time of sixteenth 
observation 

43.21 

.700 

43.26 

.701 

36 

Immediately after 
executing second 
velocity correction 

9-21 

.089 

9.23 

.089 

64 

At time of twenty- 
eighth observation 

18.55 

.113 

18.61 

.114 

72 

Immediately after 
executing third 
velocity correction 

6.94 

.054 

6.98 

.054 

74 

At time of thirty- 
sixth observation 

7.36 

.077 

7.4 o 

.078 

78,847 

At end point 

1.71 

.163 

1.79 

.170 
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TABLE IV.- MAGNITUDE OF r, KM 


At time of 

l6 bits 

l6-bit A matrix 

l6 bits 

27-bit A matrix 

Resumption of observations after 
first velocity correction 
(24 hours) 

216 

201 

Resumption of observations after 
second velocity correction 
(64 hours) 

363 

72 

Resumption of observations after 
third velocity correction 
(74 hours) 

259 

299 

End point (78.847 hours) 

1177 

571 
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TABLE V - COMPARISON OF ACTUAL AND ESTIMATED TRAJECTORY POSITION DEVIATIONS FROM THE REFERENCE 

TRAJECTORY 
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Actual velocity correction 
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Figure 1.- Block diagram of complete system. 
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Figure 6.- Position error ellipsoid before and after moon observation 20. 
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Figure 7.- Variation of |AVq| with word lengths of l6, l8, and 27 hits. 
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conducted so as to contribute ... to the expansion of human knowl- 
edge of phenomena in the atmosphere and space. The Administration 
shall provide for the widest practicable and appropriate dissemination 
of information concerning its activities and the results thereof ” 

— National Aeronautics and Space Act of 1958 
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